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The current study was designed to test Lathrop and Richmonds 
conclusion that perceived learning is an important factor in faculty evaluations. A 17 
item questionnaire including the perceived learning area was administered to 470 
students (2 classes in each of 8 departments) at Chico State College. Separate 
multiple regression equations for the prediction of instructor ratings and for course 
evaluations indicated that perceived learning is one of three factors significantly 
influencing these ratings. A supplementary finding was that students can rate unit 
factorial items with relative independence. The results constitute strong support for 
Lathrop and Richmond’s conclusion. A theoretical framework for student’s ratings of 
courses and instructors is advanced. (Author) 
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UNIT FACTORIAL RATINGS BY COLLEGE STUDENTS 
OF COURSES AND INSTRUCTORS 
Richard G. Lathrop 
Chico State College 
Abstract 

The current study was designed to test Lathrop and 
Richmond’s (1967) conclusion that perceived learning is an 
important factor in faculty evaluations. A 17 item 
questionnaire including the perceived learning area was 
administered to 470 students (2 classes in each of 8 
departments) at Chico State College. Separate multiple 
regression equations for the prediction of instructor 
ratings and for course evaluations indicated that perceived 
learning is one of three factors significantly influencing 
these ratings. A supplementary finding was that studenrs 
can rate unit factorial items with relative independence. 
The results constitute strong support for Lathrop and 
Richmond’s conclusion. A theoretical framework for 
students’ ratings of courses and instructors is advanced. 



U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE 
OFFICE OF EDUCATION 




i ^ 

K 







,er|c 



THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION ORIGINATING IT. POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION 
POSITION OR POLICY. 



UNIT FACTORIAL RATINGS BY COLLEGE STUDENTS 



OF COURSES AND INSTRUCTORS 
Richard G. Lathrop 
Chico State College 

In a paper presented at the Western Psychological 
Association’s annual convention, Lathrop and Richmond 
(1967) discussed a factor analysis of a questionnaire 
designed to assess student reactions to college courses 
and instructors. The ratings of courses and instructors 
were included among the 72 items of that questionnaire. 

Of primary interest in their analysis was the 
identification of a factor which they termed "Perceived 
Learning." Items loading both highly and uniquely on 
this factor were: (1) My over-all knowledge of the 

subject matter has increased, (2) I have improved in my 
ability to recognize significant facts and interpret 
information, (3) I have improved in my ability to express 
nyself in this subject, (4) I have learned to think 
through problems, analyze questions, and reach 
conclusions, and (5) I have learned interrelationships 
of facts and ideas. Thus, this factor appeared to indicate 
what the student, himself, felt that he had gained from 
the course in terms of content. Of particular importance 
to the current study was the finding that the overall 
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course evaluation item had its only significant loading 
on this Perceived Learning factor. In addition, the 
evaluation item had its highest loading on that factor . 

The validity of these findings could he criticized 
on t'vfo grounds. First, 27 of their 72 items were 
specifically chosen to reflect perceived, learning and 
the remainder were sampled from factors shown hy other 
studies to tap areas of differences in student opinion 
about either instructors or courses. Perhaps the overall 
evaluation items loaded on the majority factor of perceived 
learning just due to response bias. Second, the sample 
of students tested by Lathrop and Richmond represented a 
very small part of the total student population (psychology 
students only). Perhaps the Perceived Learning factor is 
unique to this non-representative group. The current 
stud.y attempts to control for both criticisms. 

Method 

Sub j ect s . — ^70 students in two classes from each of 
eight different departments at Chico State College 
completed the questionnaire. The specific classes and 
instructors were chosen on the basis of a prior campus-wide 
teacher evaluation survey. Three judges indicated those 
departments in which an excellent and a poor teacher 
were teaching approximately equivalent courses. Prom 
this list, those departments were eliminated in which: 

(1) One of the two instructors refused to participate, 
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(2) Both instructors were not teaching courses designed 
for majors in the department, or ( 3 ) Class sizes were not 
within the range of 20 to 40 registered students. The 
smallest of the l6 classes returned 1? questionnaires; 
the largest returned 32 . 

Evaluation instrument . — To eliminate the possibility 
of response bias due to overloading of items specific to 
a single factor, the basic instrument contained only 11 
items. Each of these unit items was constructed from 
the sum of 5 individual questions loading on factors 
shown by other stud.ies to be important in the evaluation 
of teaching. Thus each item was concerned with a single 
factor as a unit. In addition, course description, 
overall course evaluation, instructor evaluation, and 
three grade-point indices were included in the final 
instrument. Thus, Lathrop and Richmond's Perceived 
Learning factor was represented by a single item and 
constituted less than 6 % of the questionnaire. In 
addition to the quantitatively scored items, space was 
provided for a written description of the instructor 
and also for criticisms of the evaluation instrument. 

(A copy of this instrument is included as a supplement 
to this report.) 

Procedure ; — Each class was administered the questionnaire 
during a regularly scheduled class meeting by the author, 
or his research assistant. The classes' instructors were 
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not present during the period of testing. The instructions 
on the questionnaire were read to the class with special 
emphasis placed on the confidentiality of the results and 
upon the instruction that the students were to rate the 
global meaning underlying the cluster of sub-parts to each 
item. The ratings were completed by each student present 
and collected by the investigators prior to permitting the 
instructor to return to the class. 

Results 

All of the quantitative portions of evaluation 
instrument were analyzed by an IBM 1620 computer. The 
verbal descriptions of the instructor were grouped and 
typed for return to the instructor. AH criticisms of the 
questionnaire were collated and used for the revising of 
the questionnaire attached to this report. 

Of particular importance for the current report are 
the two separate multiple regression analyses using the 
over-all course evaluation as criteria to be predicted. 

These results are summarized in Table 1 reporting 

Insert Table 1 about here 

significant loadings. As noted there, only two items 
weighted significantly in the prediction of instructor 
ratings: Dynamic, Enthusiastic Instructor and Perceived 

Learning. Together they account for 66 » 7 % (R = *817) of 
the variability in individual instructor ratings. Three 
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ite^ns weighted significa.ntly in the prediction of course 
evaluations: Perceived Learning, Encouragement of Inquiry, 

and Dynamic, Enthusiastic Instructor. These three items 
account for 55 . 7 % (R = .746) of the variability in individual 
course evaluations. 

The basic correlation matrix for all items except 
the two composite ratings yielded information concerning 
the effectiveness of unit factorial ratings. Variables in 
this matrix were reflected to yield a positive manifold 
and mean correlations of each variable with every other 
variable were obtained. The average of these mean 
correlations was +.194 (0^ = .093) for the matrix as a 
whole. Thus relative independence can be demonstrated. 

Discussion 

The results of the multiple regression analyses 
constitute extremely strong support for Lathrop and 
Richmond’s contention that Perceived Learning is a very 
important factor influencing college students’ evaluations 
of both courses and instructors. In view of the differences 
in evaluation instruments and populations sampled, the 
agreement of the two studies is striking. 

The distinction between Perceived Learning and objective 
measures of classroom performance, however, should be 
emphasized. The variable Anticipated Class Grade - an 
"objective” (?) measure of classroom performance improves 
the R of course evaluation prediction by .0019 over the 
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more predictive items. Similarly, it improves the 
for instructor rating prediction by only .►0020. Perceived 
Learning, then, is the somewhat nebulous cognitive feeling 
on the part of the student that he has improved as a function 
of participating in the course. 

The other two predictive items (Dynamic, Enthusiastic 
Instructor and Encouragement of Inquiry) lead to a general 
theoretical structure of student ratings of courses and 
instructors. It would appear that an enthusiastic instructor 
is probably able to communicate that enthusiasm to his 
students. Placed together, these three items constitute a 
truly idealistic view of what the learning process really 
should be. In the best of all possible worlds, the student 
would have an enthusiastic inquiry into issues and concepts 
which results in deep personal growth. Such a world 
would not be clouded by concern over grades, the mentor’s 
personality characteristics or pedagogical mannerisms. 

It is interesting that, from our sophisticated (and 
perhaps jaded?) level as instructors, we have been concentrating 
on measuring just those factors - GPA, personality, and 
teaching habits ~ which we hope have no influence on 
learning. Perhaps if we adopt the idealistic view of the 
student, we can improve our assessment techniques and, 
just possibly, our teaching. 
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Table 1 



Summary of Prediction Weights 



;^io 



Criterion 



Overall Course Evaluation 



Instructor Rating 



Item 1 
r2 



Perceived Learning 



.4300 



Dynamic, Enthusiastic 
Instructor 
.5981 



Item 2 



R" 



Item 3 
R^ 



Encouragement of Inquiry 



.5172 



Dynamic, Enthusiastic 
Instructor 

.5573 



All items (15) 



R^ 



.5844 



Perceived Learning 



.6667 



.7181 
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Research Questionnaire 






Class // 



Faculty Evaluation 



Test // 



This questionnaire has been prepared for the purpose of gathering data to be 
used in the construction of next year’s faculty evaluation. Approximately 30 
classes here at Chico State College will be administered this questionnaire. You 
will not be identified nor will the actual questionnaire be returned to your 
instructor, so please answer every question as honestly as possible. 

1. Class description: Lecture Lecture-discussion 



Activity Other (please state:) 



Instructions 

The following questions have been selected on the basis of a previous research 
study conducted at Chico State College, In that study, it was found that the indi~ 
vidual questions tended to group together in clusters. In most of these tlusters, it 
was apparent that a particular trait vas being tested. The items on this test, then, 
give the group of questions which test a single trait. You are asked to read each 
item carefully, determine the particular trait that seems common to all parts of the 
item, and then enter your rating for this instructor in the space provided. Please 
take your time and make an accurate rating. 



Use the following scale for all of your ratings : 



1 2 


3 


4 


5 


6 


7 


8 9 0 


Less than 






Average 






More than / 


in any 






College 






in any / 


other class 






Class 






other class / 
/ 

> 














r 

/ 

Not 














Applicable 



2. My over-all knov/ledge of the subject matter has Increased j I have Improved 
in my ability to recognize significant fact and Interpret information, in ray ability 
to express myself in this subject; I have learned to think through questions, analyze 
problems, and reach conclusions ^ and I have learned interrelationships of facts and 
ideas. 



Rating ! 

3. There are discussions between teacher and students as opposed to responses 
to questions; at times the Instructor contributes to student discussions; at times 
the students take over direction of the class; contrasts with teacher asking students 
for interpretations, explanations, hypotheses ^ or theories about material in course, 
also contrasts xjith instructor limiting class discussion to topics directly and 
clearly related to the subject matter. 

Rating i_ t 

o , — - - 
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4. The material presented is devoted to statements of the instructor's personal 

• opinions; he attemots to "shock" the students as a means of stimulating discussion 
and/or interest; he presents material which is in strong opposition to commonly 
''accepted attitudes or opinions; his manner is casual and disrespectful; he refers 
to himself or his own opinions. 

Rating / ! 

5. The assigned reading is clearly appropriate to the field or area under study; 
examples used by the teacher are illustrative of the topic being discussed; assigned* 
papers or homework are directly related to the area of study; the tests are appropri- 
ate for the material covered; and the papers and tests in this course contribute to 
the learning experience. 



Rating i / 



6. The Instiactor is a dynamic person; is enthusiastic about this course and 
its material; speaks with expressiveness and variety in tone of voice; contrasts 
with instructor appears to be disinterested in this course; also contrasts with 
instructor is incoherent and/or vague in his presentation. 

Rating / ! 

7. The instructor attempts to alleviate tensions in difficult situations to 
keep students from feeling embarrassed or uncomfortable; he asks students for their 
personal opinions; he treats students as his equal; he seems to understand student 
comments and questions even when they are not clearly expressed; contrasts with 
instructor appearing to be cold towards or unaware of students. 



Rating / ! 

3. The instructor asks for more than students can get done; he assigns very 
difficult reading; he has assigned a great amount of reading; contrasts with assign- 
ments in this class are easily completed; also contrasts with insufficient amount of 
work assigned to cover the material of the course. 

Rating ! 

9. The class is well organized; the instructor is objective in evaluating 
students* performance in this class; he speaks with great precision with apparent 
concern for the accuracy of details; he appears to have a well-developed plan for the 
course; a student in this class knows where he stands at any time. 

Rating / f 

10. The instructor encourages inquiry in this class; the course 'opens up* 
rather than closes issues for discussion; I have been stimulated to strive for 
excellence; I have developed a desire to find out things; my Interest in this subject 
has grown. 




Rating / 



11. The Instructor is ill at ease when talking to students; he seems hesitant 
nrtmsure of himself; he shows signs of nervousness; contrasts with teacher seems 
comfortable and relaxed in the classroom; also contrasts with instructor seems con- 
fident of his oxm knowledge and ability to communicate it to the student. 



Rating / / 

12. I have increased my understanding of the relationship of this course to 

others; this course has increased my ability to prepare myself in this area; it has 
assisted in my preparation for my vocation; I have discovered my possibilities and 
deficiencies in this Jfield; 'and 1 have discovered where to place special emphasis in 
my preparation for this field. 

Rating / / 

13. How would you rate your Instructor in general (all-around) teaching ability? 

(1) Poorest instructor I have ever had. 

(2) Much poorer than the average instructor. 

(3) Poorer than the average instructor. 

(4) Slightly poorer than the average instructor. 

(5) About the same as the average instructor. 

(6) Slightly better than the average instructor. 

(7) Better than the average instructor. 

(S) Much better than the average instructor. 

(9) Best instructor I have ever had. 

Rating / / 

14. Hov 7 x7ould you rate the over— all value of this course? 

(1) Poorest course I have ever had. 

(2) Much poorer than the average class. 

(3) Poorer than the average class. 

(4) Slightly poorer than the average class. 

(5) About the same as the average class. 

(6) Slightly better than the average class. 

(7) Better than the average class. 

(8) Much better than the average class. 

(9) Best course I have ever had. 

! i 



15. In one word or phrase^ how would you characterize your instructor's 
outstanding characteristic? 



Some further information is desired to help complete this study. Please answer 
every item. It is important to know both your estimated course grade and the 
grade-point average of your entire college career. 

Again, place the appropriate number from the scale below in the box at the right. 



(1) 


A 


3.75-4.00 


(2) 


A-B 


3.25-3.75 


(3) 


B 


2.75-3.25 


(4) 


B-C 


2.25-2.75 


(5) 


C 


1.75-2.25 


(6) 


C-D 


1.25-1.75 


(7) 


D 


.75-1.25 


(3) 


D-F 


.25- .75 


(9) 


F 


.00- .25 



16. 


Your estimated grade in this course. 


/ 


/ 


17. 


Your over-all GPA in college (estimate). 


/ 


. / 


18. 


(Please leave this space blank) 


/ 


. ,/ 



19. As a final item in the questionnaire, would you please list any area that we 
may have forgotten to cover in our original list of traits relevant to the general 
evaluation of college instructors and their courses? 



